Papers with speech tokenization
Scaling Properties of Speech Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Speech Language Models (SLMs) aim to learn language from raw audio without textual resources. |
| Approach: | They propose to use scaling properties of neural language models to estimate scale at which SLMs will be trained . they establish a strong correlation between pre-training loss and downstream syntactic and semantic performance . |
| Outcome: | The proposed model will have the English proficiency of text-based Large Language Models. |
DM-Codec: Distilling Multimodal Representations for Speech Tokenization (2025.findings-emnlp)
Copied to clipboard
Md Mubtasim Ahasan, Md Fahim, Tasnim Mohiuddin, Akmmahbubur Rahman, Aman Chadha, Tariq Iqbal, M Ashraful Amin, Md Mofijul Islam, Amin Ahsan Ali
| Challenge: | Existing speech tokenization models lack contextual representations for speech synthesis . absence of contextual representation results in elevated WER and WIL scores . |
| Approach: | They propose a language model-guided distillation method that incorporates contextual information into a comprehensive speech tokenizer. |
| Outcome: | The proposed method outperforms state-of-the-art tokenization models in reducing WER and WIL scores. |